(N 

o 

-i— > 
O 

O 



Clusters, generating functions and asymptotics 
for consecutive patterns in permutations* 



Sergi Elizalde ' Marc Noy 



i 



Abstract 



We use the cluster method to enumerate permutations avoiding consecutive patterns. We 
reprove and generalize in a unified way several known results and obtain new ones, including 
some patterns of length 4 and 5, as well as some infinite families of patterns of a given shape. By 
enumerating linear extensions of certain posets, we find a differential equation satisfied by the 
inverse of the exponential generating function counting occurrences of the pattern. We prove 
' that for a large class of patterns, this inverse is always an entire function. 

. We also complete the classification of consecutive patterns of length up to 6 into equivalence 

classes, proving a conjecture of Nakamura. Finally, we show that the monotone pattern asymp- 
totically dominates (in the sense that it is easiest to avoid) all non-overlapping patterns of the 
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same length, thus proving a conjecture of Elizalde and Noy for a positive fraction of all patterns. 

1 Introduction 

In this paper we use the cluster method of Goulden and Jackson in order to obtain new results on 
the enumeration of permutations avoiding consecutive patterns. Recall that a permutation tt avoids 
a consecutive pattern a if no subsequence of adjacent entries of ir is in the same relative order as 
the entries of a. Given a pattern a, the cluster method consists of counting partial permutations in 
which each element is involved in at least one occurrence of a, the so-called clusters. By inclusion- 
exclusion, the enumeration of clusters provides the enumeration of permutations according to the 
number of occurrences of a. 

Counting clusters can be seen as counting linear extensions in a certain poset. For instance, if a 
^ ■ is the monotone pattern, the corresponding poset is simply a chain, and counting linear extensions 

is a trivial task. In fact, not only the monotone pattern 12 ■ ■ ■ m can be analyzed in this way, but 
also the pattern 123- • • (s — l)(s + l)s(s + 2)(s-|-3) • • -m (Corollary [23]), and other related patterns, 
which we call chain patterns (Theorem 12.4ft . Another significant case is that of non-overlapping 
patterns a, which are those for which two occurrences of a in a permutation cannot overlap in more 
than one position. The associated poset is not difficult to analyze when <x\ = 1, in which case the 
number of permutations avoiding a 6 S m depends only on the value b = a m , and we can derive 
a linear differential equation (Theorem 13. ip satisfied by the inverse of the associated exponential 
generating function. A weaker version of this result was proved in [7] using representations of 
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permutations as binary trees. For chain patterns and non-overlapping patterns, the differential 
equations that we obtain can also be deduced from the work of Khoroshkin and Shapiro p2] . 

A more intricate example is the pattern 1324. This case was left open in [7] and cannot be 
solved with the techniques from [12] either. The number of linear extensions of the associated poset 
is related to the Catalan numbers, and we prove that the inverse of the generating function for 
this pattern satisfies a linear differential equation of order five with polynomial coefficients. Again, 
the technique can be extended to cover the general pattern 134- • • (s + l)2(s + 2)(s + 3) ■ ■ ■ m 
(Theorem 14.2ft . For other patterns of length 4, namely 1423 and 2143, we find recurrence relations 
satisfied by their cluster numbers (Section [5]), which already appeared in [3], but we are not able to 
find closed solutions in terms of differential equations. In fact, we conjecture that the inverse of the 
generating function for permutations avoiding 1423 is not D-finite. If true, this conjecture would 
give the first instance of a pattern with this property, and it would make a related conjecture of 
Noonan and Zeilberger for classical patterns less believable. 

The present situation for small patterns is the following. Say that two patterns are equivalent 
if their numbers of occurrences in permutations have the same distribution. There are two inequiv- 
alent patterns of length 3, already solved in [7J. There are seven inequivalent patterns of length 
4, four of which are solved now, but we still do not have closed solutions for 1423, 2143 and 2413. 
There are 25 inequivalent patterns of length 5. Four of these are easily solved with the techniques 
from [7J, and we can now solve four additional ones, namely 12435, 12534, 13254 and 13425. The 
remaining 17 patterns (which include 2 non-overlapping ones) are unsolved in terms of closed solu- 
tions or differential equations. For patterns of length 6, we prove four conjectures of Nakamura |15| 
regarding the equivalence of certain pairs, completing the classification into equivalence classes, 
and proving that there are exactly 92 inequivalent patterns. 

Regarding asymptotic enumeration of permutations avoiding a pattern, we prove that the mono- 
tone pattern dominates all non-overlapping patterns of the same length (Theorem l6.8p . thus proving 
a special case of a conjecture by Elizalde and Noy [7]. It was shown by Bona [2 J that the number 
of non-overlapping patterns is asymptotically a positive fraction of all patterns. We also show that 
the inverse of the generating function of permutations according to the number of occurrences of a 
given pattern is an entire function in several important cases (Theorem l6.ip . but not for the pattern 
2143. 

We conclude this introductory section with definitions and preliminaries needed in the rest of 
the paper. In Sections [2] and [3] we study monotone and non-overlapping patterns, and related 
patterns. Section H] is devoted to the pattern 1324 and generalizations, and Section [5] to some other 
patterns of length 4. In Section [6] we present our asymptotic and analytic results. We end the 
paper with some open problems. 

1.1 Consecutive patterns 

Given a sequence of distinct positive integers r = t\ ■ ■ -t^, we define the reduction st(r) as the 
permutation of length k obtained by relabelling the elements of r with {1, . . . , k} so that the order 
relations among the elements remains the same. For instance st(46382) = 34251. Given permuta- 
tions 7r 6 S n and a 6 S m , we say that it contains a as a consecutive pattern if st(7Tj • • • 7Tj +m _i) = a 
for some i G {1, . . . , n — m + 1}. We denote by c a (ir) the number of occurrences of a in ir as a 
consecutive pattern, and by ot n {a) the number of permutations in S n that avoid a as a consecutive 
pattern. In the rest of the paper, the notions of occurrence, containment and avoidance always 
refer to consecutive patterns, even if it is not explicitly stated. 
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Let 

n>0 7re5 n 

be the bivariate exponential generating function for occurrences of a in permutations. It is conve- 
nient to define u a (u,z) = 1/ P a (u, z). Note that 

1 z n 

is the generating function of permutations avoiding a. We drop the subscript a from P and w when 
the pattern is clear from the context. If £ is a set of patterns, we define Pj](u,z) and ljj^(u,z) 
similarly, where u marks the total number of occurrences of all the patterns in S. 

Two occurrences of a in a permutation may overlap in certain positions. This is a basic issue in 
what follows and motivates the following definition. Let O a be the set of indices i with 1 < i < m 
such that st(<Tj + iO"j + 2 . . . cr m ) = st(<7i<72 . . . a m -i). Equivalently, i G O a if there is some permutation 
in 5 m .f j where both its leftmost m entries and its rightmost m entries form occurrences of a (these 
occurrences overlap in exactly m — i positions). We call O a the set of overlaps of a. Note that if 
m > 2, then m — 1 € O a . 

We say that two patterns a and r are strongly c-Wilf- equivalent, or simply, that they fall in 
the same class, if P a (u,z) = P T (u,z). While this implies that a n (a) = a n (r) for all n, it is an 
open question to determine whether the converse holds. Given a pattern o\ ■ ■ ■ a m , its reversal is 
o~m ■ ■ ■ ci) and its complementation is (m + 1 — o\) • • • (m+ 1 — a m ). Reversal and complementation 
do not change the equivalence class of a pattern. Patterns of small length were first studied in [7]. 
It was shown that patterns of length three fall into two classes, represented by 123 and 132, and 
the associated generating functions P a (u,z) were computed explicitly. Patterns of length four fall 
into seven classes, represented by 1234, 2413, 2143, 1324, 1423, 1342 and 1243. Three of them, 
namely 1234, 1342 and 1243, were solved in [7] in terms of the generating functions. Let us remark 
that the situation is quite different for classical patterns, that is, patterns that appear in non- 
necessarily consecutive positions of a permutation. For instance, all permutations of length 3 are 
Wilf-equivalent in the classical setting, due to a non-trivial bijection between permutations avoiding 
123 and those avoiding 132. 

1.2 The cluster method 

The cluster method of Goulden and Jackson |1CH [XT] is a powerful method for enumerating words 
with respect to occurrences of certain substrings, based on inclusion-exclusion. Several extensions 
and implementations of the method have been given in the literature, most notably in [IT]. Let 
us now summarize an adaptation of the cluster method to the enumeration of permutations with 
respect to the number of occurrences of a consecutive pattern. This adaptation has been recently 
used in [3], and it has many similarities with a method of Mendes and Remmel [H] based on the 
combinatorics of symmetric functions. 

For fixed a G S m , a fc-cluster of length n with respect to a is a pair (ir, (i\,i2, • • • , ik)) where 

• 7T G S n , 

• 1 = i\ < i2 < • • • < ik = n — m + 1, 
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• for each 1 < j < k, st (7^.7^+1 . . . n ij+m -i) = a, 

• for each 1 < j < k — 1, ij + \ < ij + m — 1. 

In other words, the are starting positions of occurrences of a in tt, all the entries of tt are in 
at least one of the marked occurrences, and neighboring marked occurrences overlap. Note that 
ij+i — ij 6 O a (the overlap set) for all j, and that tt may have more than k occurrences of a. 
Sometimes we write (tt; i\, ii-, ■ ■ ■ , ik) instead of (tt, (i\,i2, • • • , ik))- For example, if a = 1324, then 
(142536879; 1, 3, 6) is a 3-cluster of length 9, since 1425,2536 and 6879 are occurrences of a and 
all the entries are in at least one of the occurrences. Notice that the 1425 and 2536 overlap in two 
positions, whereas 2536 and 6879 overlap only in one position. 

Let r n fc be the number of fc-clusters of length n with respect to a fixed a. We denote by 

R a (t,z) =Vr n /- 

n,k 

the exponential generating function (EGF for short) for clusters, and by 

R a (t,x) = ^r ntk t k x n 

n.k 

the corresponding ordinary generating function (OGF for short). 

The following theorem, which is an adaptation of \\.\\ Theorem 2.8.6] to the case of permutations, 
expresses the EGF for occurrences of a as a consecutive pattern in permutations in terms of the 
EGF for clusters. We include a short proof for completeness. 

Theorem 1.1 QllJ). For any pattern a we have 

u a (u, z) = 1 — z — R a (u — l,z). 
In particular, the EGF for a -avoiding permutations is 

= Z^(M = l- z -R a (-l,z)- 

Proof. A permutation tt can be seen as a sequence of consecutive blocks, where each block consists 
either of elements all belonging to an occurrence of a, or of an element not belonging to any occur- 
rence of a. The first kind of block is a cluster and is encoded by the generating function R a (—1, z) 
(since setting u = — 1 marks, by inclusion-exclusion, the exact number of occurrences of a), and 
the single elements are encoded by z. Notice that the fact that occurrences in a cluster overlap 
guarantees the uniqueness of the decomposition. Finally, the sequence construction corresponds to 
1/(1 — (z + R a (— 1, z))), thus proving the result. □ 

We usually denote A(t, x) = 1 — x — R(t, x) and A(t, z) = 1 — z — R(t, z), so that u) a (u, z) = 
A(u - l,z). 

Theorem 11.11 reduces the study of the distribution of occurrences of a pattern in permutations 
to computing the cluster numbers r n ^- We now show that these numbers can be expressed in terms 
of linear extensions of certain posets. Fix a G S m . For given n, k, let 

z n,k = {(n^2, • • • ,ik) ■ k = Mfc = n - m + 1, and i j+ i - ij <E O a for 1 < j < k - 1}, (1) 
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and fix . . . ,i k ) £ 1° k . A permutation ir G S n has the property that (n; i\, . . . , ik) is a /c-cluster 
of length n if and only if, for each 1 < j < k, 

St(vTj j VTj j + 1 . . . 7Tj j+m _i) = a. (2) 

If we denote by ? G 5 m the inverse of <r, so that is the position of £ in a, then ([2]) is equivalent to 

TTft+ij-l < 7Tf 2 +i 3 -l < • • • < 7T? m +i 3 — 1- ( 3 ) 

The conditions ([3]) for 1 < j < k define a partial order on the set {iri, 7T2, . . . , 7r„}. We denote the 
corresponding partially ordered set (poset) by P^ ix i . If we denote by C(P) the set of linear 
extensions (i.e., compatible linear orders) of P, then it follows that (jr;ii, . . . ,it) is a fc-cluster of 
length n with respect to a if and only if tt G C(P^ tii ■). We denote by V° k the multiset of 
such posets for all values of (ii, . . . G ^nk- Note that some posets in V^ k can appear with 
multiplicity, as in the cases discussed in Section [2j Alternatively, we could mark the elements 
7Ti x , 7Tj 2 , . . . , 7Tj fc in - , to ensure that all the posets in V° k are different as marked posets. We 

have that 

r n , k = M p )\- ( 4 ) 

We also define the multisets = Ufc>i Pn,k and = U n • 

Finally, note that for the reversal or complementation of a pattern a, the corresponding clusters 
are also reversed or complemented, the set O a does not change, and the posets that we obtain are 
isomorphic to those for a. 

1.3 Ordinary and exponential generating functions 

Here we describe a tool that we use to switch between the OGF and the EGF of a sequence. Let 
L be the linear operator on formal power series such that L(x k ) = 4j for all k > 0. 

Lemma 1.2. Let A(x) = ^ n >QC n x n be an OGF and let A(z) = X^n>o an fiT ^ e corres P° n ding 
EGF. Let L denote the integral operator with respect to z, that is, LF(z) = J F(v) dv, and let 
j > 0. Then 

1. L(x j A) = Li A; 

2. L{x j+1 A') = Li(zA'); 

3. L(xiA^) = z?A®. 

Proof. All the properties are easy to prove. For instance, claim 1 follows from 

L&A) = L (£ a n x^) = £ a^-^ = PA. 

□ 

Given a linear differential equation for A{x) with polynomial coefficients, Lemma 11.21 will be 
used to obtain a linear differential equation for A(z) with polynomial coefficients. 

Let us finish this section with a remark about notation. Throughout the paper we deal with 
differential equations for multivariate generating functions. All the derivatives that appear are 
always partial derivatives with respect to z. Similarly, initial conditions are for z. 
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2 Monotone and related patterns 
2.1 The pattern a = 12 ... m 

For the monotone pattern a = 12 ... m, a differential equation satisfied by uj a (u, z) was given in [7J 
Theorem 3.1]. The proof is based on representations of permutations as increasing binary trees. 

Theorem 2.1 ([7j). Let m > 3, let a = 12 ... m, and let oj(z) := uj a (u, £). Then u is the solution 
of 

+ (1 _ u )( w (m-2) + . . . + a/ + W ) = (5) 

iwtft w(0) = l,a/(0) = -l,w (i) (°) = /or 2 < i < m - 2. 

To warm up for the more complicated patterns in upcoming sections, let us give an alternative 
proof of Theorem 12.11 using the cluster method. It is clear that O a = {1,2, ... ,m — 1}, so for 
vr G S n , (jr;ii,... ,ik) is a /c-cluster with respect to o if and only if tk\ < 1T2 < • • • < vr n and 
1 < ij+i — ij < rn — 1 for all j. It follows that the OGF for the cluster numbers is 

/ ^ ^ 

R a {t,x) 



l-t(x + x 2 -\ hi™" 1 )' 

and so 

~ — ^ — — 

2) = 1 — x — R a (t, x) = -. k tt- (6) 

y J K ' 1 -t(x + x 2 H hx" 1 - 1 ) v ; 

Clearing denominators in ([6]), applying the transformation L, and using Lemma ll.2l we get that 

(1 - t(I + I 2 + ■ ■ ■ + Z ) = 1 - z - tz. (7) 

Differentiating m — 1 times we obtain a differential equation for A(t, z): 

A (m-1) _ t ( A (Fi-2) -I \- A' + A) =0, 

with initial conditions A(O) = 1,A'(0) = -1, and A®(0) = for 2 < i < m - 2. Equation © is 
now obtained making the substitution t = u — 1 and using Theorem ll.il Note that for m = 2, the 
right hand side of ([7]) does not cancel, but it produces a term — 1 — t, from where one can recover 
the well-known generating function for the Eulerian numbers. 
In fact, we see from equation © that 

1 - x - R a (-l,x) = if^ = J2 * jm ~ E * J ' m+1 > 

i>o i>o 

so 

~jm ~jm+l 

w '(°' z ) = E " E 7-— W (8) 

^ ?m ( 7771 + 1)! 

i>o u ' j>o u ; 
recovering a formula from |llj . Equivalently, 

where Aj = e - ^ - are the mth roots of unity. 
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2.2 Chain patterns 

The ideas in the above proof of Theorem 12.11 extend to any pattern a for which the poset satisfied 
by the entries of tt in every cluster is a chain. Recall that a poset is a chain if it is a linear order, 
that is, all its elements are comparable. 

Definition 2.2. We say that a G S m is a chain pattern if all the posets in V a are chains. 

Examples of chain patterns are given in Subsection 12.31 The following result shows that the 
condition of being a chain pattern is quite restrictive. 

Lemma 2.3. Let m > 3, and let a G S m be a chain pattern. Then a (or one of the permutations 
obtained by applying the reversal and/or complementation operations) satisfies that a\ = 1, o<i = 2 
and o~ m = m. In particular, m — 2 G O a . 

Proof. Recall that always m — 1 G O a . By hypothesis, the poset P^m-limi corresponding to 
clusters consisting of two occurrences of a overlapping in one position, is a chain. Denoting by 
? G S m the inverse of a, this poset is determined by the inequalities 7T S x < 7r f2 < • • • < ir<; m and 
7T?i+m-l < 7r ? 2 +m-i < •■ < 7T? m +m-l- Since ir m appears in both lists, it must be the rightmost 
element of one list and the leftmost one of the other, from where {o~i,o~ m } = {l,m}. By applying 
reversal if necessary, we can assume that a\ = \ and a m = m. Since u\ < 02 and a m -\ < cr m , we 
have that m — 2 G O a . 

Consider now the poset P^m— 2 1 m— 1> corresponding to clusters consisting of two occurrences of 
a overlapping in two positions, which again is a chain. This poset is determined by the inequalities 
7Ti < 7r ?2 < • • • < 7r Cm _ 1 < 7r m and 7r m „i < 7T f2+m _ 2 < • • • < 7T ?m _ 1+m _ 2 < 7r 2m _ 2 . For this poset 
not to have incomparable elements, we must have either ? m _i = m — 1 or q + m — 2 = m, which 
is equivalent to <r m _i = m — 1 or <T2 = 2. □ 

Now we state the generalization of Theorem 12.11 to chain patterns. 

Theorem 2.4. Let m > 3, and let a G S m be a chain pattern. Let uj(z) := 0J a (u, z). Then uj is the 
solution of 

J m -^ + (l-u)J2 uj( m - d -V = (9) 
deO a 

with w(0) = (0) = -l,wW(0) = for 2 < i < m - 2. 

Proof. For each fixed tuple {i\, 12, ■ ■ ■ , ik) G 1° k , as defined in ([T|), there is a unique permutation 
tt G S n such that (71"; ii, . . . , ik) is a cluster with respect to a. Indeed, ix is the unique linear extension 
of the poset i 3 ^ ti i 2 ... i > as defined in Section [L2"l which is a chain by hypothesis. Thus, the number 
of fc-clusters of length n is the number of elements of so the corresponding OGF is 

- tx m 

1 1 l^deOa X 

and so A(t, x) = 1 — x — R a (t, x) satisfies the equation 

A(t, x)-tJ2 x d A(t, x) + t^2(x d - x d+1 ) + tx m + x-l = 0. (10) 
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Now we apply the transformation L to (jlOp and use Lemma 11.21 Differentiating m — 1 times and 
using that {m — 2, m — 1} C O ct , which we showed in Lemma 12.31 we get a differential equation for 
AO) = A(t,z): 

A (m-1) _ t ^ ^(m-d-1) = Q) 

with initial conditions A(0) = 1, A'(0) = -1, and A®(0) = for 2 < j < m - 2. A differential 
equation for uj is now obtained making the substitution t = u — 1 and using Theorem 11.11 □ 

2.3 Examples 

A good example is the pattern 123 . . . (s — l)(s + l)s(s + 2)(s + 3) . . . m, for arbitrary s > 3 and 
m > s + 2. By reversal and complementation, we can assume without loss of generality that 
m — s < s. In this case, O a = {s, s + 1, . . . , m — 1}. 

Corollary 2.5. Let s > 3 and 2 < m - s < s, let a = 123 . . . (s - l)(s + l)s(s + 2)(s + 3) . . . m, 
and let co(z) := uj a (u,z). Then oj is the solution of 

with w(0) = l,a/(0) = -l,u®(0) = for 2 < i < m - 2. 

Corollary 12.51 for s = 3 and m = 5 states that ^i2435( , u, z) satisfies the differential equation 

w (4) + (l-n)(w / + w) = 0. 

We can write 

„ , N /^a 3 -a 2 + l- n \ 

Wu,^) = ^E 4a3 + 1 _ n , 

where the sum is over the four roots a of the characteristic polynomial x 4 + (1 — + 1). For 
t = —1, the OGF for the cluster numbers i?i2435( — 1, x) = — x 5 /(l + x 3 + x 4 ) has been obtained 
automatically by Baxter, Nakamura and Zeilberger's Maple package pQ. 

A more general family of chain patterns can be obtained as follows. Given a £ S m , let 
r > be the largest index such that o-\o~i . . . o~ r = 12 ... r, let s > be the largest such that 
a m - s+ \ . . . a m -ia m = (m — s + 1) . . . (m — l)m, let a > 1 be the largest such that o\02 ■ ■ ■ o a is 
increasing, let 6 > 1 be the largest such that a m _b+i ■ ■ ■ o~m-\°~m is increasing, and let c = min{a, b}. 

Corollary 2.6. Let a G 5 m \{12 . . . m}, and let r, s, a, 6, c be defined as above. Suppose that r, s > 1 
anc? r + s > c + \, and that O a n {1, 2, . . . , m — c — 1} = (i.e., a can only overlap with itself at 
the initial and final increasing runs). Let oj{z) := cj a (u,z). Then cu is the solution of 

Jm-l) + (1 _ M )( w (c-i) + . . . + a/ + w ) = o 

with w(0) = l,a/(0) = -l,a;W(0) = for 2 < i < m - 2. 

In |15| . Nakamura conjectures that 123546 and 124536 are strongly c-Wilf-equivalent, that 
is, -Pi23546(^ ; z ) = ^124536(^5 -2)- We can prove this conjecture using Corollary 12.61 Indeed, for 
a = 123546 we have r = 3, s = 1, a = 4, b = 2, c = 2, O a = {4, 5}, while for cr = 124536 all the 
parameters are the same except that r = 2. Thus, both 0^123546(^,2) and 0J\2ibm{u, z) satisfy 

J 5) + (I - u)(uj' + u) = 

with the same initial conditions. 
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3 Non-overlapping and related patterns 



3.1 Non-overlapping patterns 

We say that a G S m is non-overlapping if O a = {m — 1}, that is, two occurrences of a cannot overlap 
in more than one position. We assume in this section that m > 2. In [5], Bona gives asymptotic 
estimates on the number of non-overlapping permutations of length m, showing in particular that 
there is a positive fraction of them. 

In [71 Theorem 3.2], the authors used binary trees to enumerate occurrences of non-overlapping 
patterns of the form a = 12 ... (6 — 1)t6, where 2 < b < m, and r is any permutation of {6 + 1,6 + 
2, . . . , m}. In fact, the formula holds for slightly more general patterns, as noted in [3], namely 
non-overlapping patterns with o\ = 1. 

Theorem 3.1 (a weaker version appears in [7]). Let a G S m be a non- overlapping pattern with 
o\ = 1, let b = o~ m , and let lo(z) := uj a (u, z). Then uj is the solution of 

m—b 

^ + (l-u) T ^—u' = (11) 
(m — b)\ 

with w(0) = l,u/(0) = -1,cjW(0) = for 2 < i < b- 1. 

Here we give a proof of Theorem 13.11 using the cluster method, different from the proof in [7]. 
It will be convenient to describe the posets in V a for an arbitrary non-overlapping pattern a. Let 
a = o\ and 6 = a m , and note that a ^ b. Without loss of generality we can assume that a < 6, 
since a and its reversal have the same cluster numbers. Since a is non-overlapping, a /c-cluster 
(it; ii, *2> • • • i ik) with respect to a has length n = k(m — 1) + 1 and satisfies ij+x — i j = m — 1 for 
all j. In the rest of this section, let n = k(m — 1) + 1. 

We have that 1° k = {(1, m, 2m — 1, 3m — 2, . . . , (k — l)(m — 1) + 1)}, and V® consists of exactly 
one poset P. To find the shape of P, denote by ? G S m the inverse of <r, and observe that the first 
m entries of n must satisfy 7r fl < 7r f2 < • • • < 7r fm to form an occurrence of a. Similarly, the entries 
in positions between m and 2m — 1 must satisfy vr fl+m _i < vr f2+m _i < • • • < vr fm+m _i. Note that 

= TTm = 7r ?Q +m-i appears in both lists of inequalities. Repeating this argument for each of the 
k occurrences of a in the cluster, we see that P is the poset in Figure [T{i). By equation r n ^ is 
the number of linear extensions of P. Figure QJii) shows another drawing of P, containing a central 
vertical chain with m + (b — a)(k — 1) elements, to which there are k — 1 chains pointing upwards 
with m — b elements each and k — 1 chains pointing downwards with a — 1 elements each. 

It is clear from the above description that the unique poset in V° depends only on a± and a m but 
not on the other entries of a, and hence so do the cluster numbers. It follows, using Theorem 1 1.1| 
that for a non-overlapping pattern a G S m , the generating function P a (u,z) depends only on o\ 
and a m . This fact was recently observed by Dotsenko and Khoroshkin [3], and independently by 
Duane and Remmel [4]. 

Proof of Theorem \3.1\ When o\ = 1, the poset P in Figure HJ^i) can be decomposed by peeling off 
the first m — 1 entries of it, which leaves another poset P' of the same shape as P whose bottom 
element corresponds to the entry 7r m (because a = 1). Of the peeled off elements, 6—1 are smaller 
than 7r m , and m — b are larger than 7r m . To produce a linear extension of P given a linear extension 
of P' , one needs to choose the relative order of these m — b elements with respect to the elements 
of P' other than 7r m . This is equivalent to choosing an (m — 6)-element multiset of the n places 
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to — b 




m — a 



a-l 



6-1 



Figure 1: Two drawings of the unique poset in V° for a non-overlapping pattern a £ S m with 
o\ = a and a m = b. 




a - 1 



where these elements could be inserted in a linear extension of P' , where n is the size (number of 
elements) of P' . If R a (t,x) = ^n r n(^) xn i s the generating function for linear extensions of P', the 
generating function for such linear extensions with m — b chosen places is 



m-b ]<{t)X 







m—b 



[m-b)\ dx m ~ b 



„m—b—l 



Rcr(t,x] 



Separating the case in which P has size m and applying the above decomposition to the other cases, 
we get a differential equation for R a : 

We can easily turn it into a differential equation for A(t, x) = 1 — x — R a (t, x), which then, applying 
the operator L and using Lemma 11.2^ becomes 



A(t,z) = 1 - z + 



(to — 6)! 



where A(t, z) = 1 — z — R a (t,z). Differentiating b times, making the substitution t = u — 1, and 
using Theorem II .1\ we get the differential equation (|11|) for u a (u,z). □ 

For the special case of b = 2, Theorem 13.11 gives the explicit expression 

v tn— 1 

LU a (u, z) = 1 — / e" u ~ hm-iy-dv, 



which can be easily checked to be the solution of equation (jlip . 
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3.2 The patterns 12534 and 13254 

Here we consider some patterns that are neither non-overlapping nor chain patterns, yet they can 
be solved using similar ideas to those in the proof of Theorem 13.11 The patterns 12534 and 13254 
are also special cases of |12|, Corollary 3.9]. 

Proposition 3.2. Let uj{z) = uji253a(u, z). Then uj is the solution of 

u/ 4) + (1 - u)z{u" + J) = 
with w(0) = l,o/(0) = -W(0) = ojW(0) = 0. 

Proof. Let us find an equation satisfied by the OGF for the cluster numbers Ri2534,(t, x). Let 
. . . ,ik) be a /c-cluster of length n with respect to 12534. If k = 1, its contribution to 
the generating function Ri2534,(t, x) is tx 5 . Suppose now that k > 2. Clearly 12 G {4,5}, since 
i2 — h 6 O12534 = {3,4}. If ?2 = 5, then the poset P^ 34 j can be decomposed as a chain 
7Ti < 7T2 < 7T4 < 7T5 < 7T3, where 7T5 is identified with the bottom element of a poset P' G V^it-i- 
Thus, the contribution of all clusters with i2 = 5 to the generating function is tx 5 ;J|-Ri2534 (t, x), 
where the derivative arises when determining the relative order of tt^ with respect to the elements 
in P' . Similarly, if 12 = 4, then -P 12 ^ 34 j consists of a chain tt\ < 1x2 < %a < ir^ < tt^, where tt^ 
and 7T5 are identified with the bottom two elements of a poset in 'P 1 1 2 _ 5 3 34 ,_ 1 . The contribution of all 

clusters with i 2 = 4 is tx 5 ^ Rl25 ^ (t ' x) . 

It follows that Ri253^(t, x) satisfies the differential equation 

^12534 (t,^) =tx 5 + tx 5 -^- + Rl2534(t,x)J . 

Writing A(t,x) = 1 — x — ^12534(^5^)5 applying the operator L, differentiating 4 times, making 
the substitution t = u — 1, and using Theorem [T7TJ we obtain the stated differential equation for 
^12534 («, z). □ 



Nakamura |15| conjectures that 123645 and 124635 are strongly c-Wilf-equivalent. We can prove 
this with an argument analogous to the above proof of Proposition [3T2] Indeed, the posets P^ ix j. 
are isomorphic for the two patterns, and both Wi23645( , u, z) and Wi24635( n > z) satisfy the differential 
equation 

J 5 ) + (1 - u)z(u" + a/) = 
with w(0) = l,w'(0) = -l,wW(0) = for 2 < i < 4. 
Proposition 3.3. Let co(z) = cui3254{u, z). Then lo is the solution of 

+ (1 - u)(u" + zu/) = 

with w(0) = l,u/(°) = -l,w"(0) = w( 3 )(0) = 0. 

Proof. In this case, O13254 = {2,4}. A fc-cluster with respect to 13254 where — ij = 2 for 
all j is a chain of length 5 + 2{k — 1). Indeed, the relationships 7ri < 7r3 < 1x2 < ?T5 < 7T4 and 
7T3 < 7T5 < 7T4 < 7T7 < 7T6 imply that tt\ < tt^ < 7T2 < 1T5 < 1T4 < ttj < ttq, and so on. Thus, the OGF 
for such clusters is % . 
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Consider now an arbitrary cluster . . ,ik) with respect to 13254. If it is not of the type 

considered above, let j be the smallest index such that ij + \ — ij = 4. The poset -P 1 ^ 254 j consists 
of a chain tt\ < ir^ < ir^ < tt^ < irj < ttq < ■ ■ ■ < ^i J+1 < ^i J+1 -i, where ^i J+1 is identified with the 
bottom element of a poset P' G "P 13254 . Note that in such a cluster, the relative order of vrj j+1 _i with 
the other elements of P' is arbitrary. From this decomposition, the following differential equation 
for -Ri3254(t, x) follows. 

tx 5 ( d - \ 

#13254^, x) = - - ( 1 + ^#13254 x) j . 

The rest of the proof is analogous to that of Proposition 13.21 □ 

We can use an argument analogous to the above proof of Proposition 13.31 to prove another 
of Nakamura's conjectures |15| . namely that 132465 and 142365 are strongly c-Wilf-equivalent. 
The key observation is that if a is either of these patterns, a fc-cluster with respect to a where 
— ij = 3 for all j is a chain of length 6 + 3(k — 1). It follows that the posets j are 

isomorphic for the two patterns, and that both 0^132465(^5 z) and ^142365(1*) z) satisfy the differential 
equation 

0/ 5 ) + {1-u){uj" + zu/) = 

with w(0) = l,w'(0) = -l,wW(0) = for 2 < i < 4. 

We remark that the ideas from this and the previous section may be used to find differential 
equations for more general families of patterns, and also for permutations avoiding more than one 
pattern. For more work in this direction, see |12| . 



4 The pattern 1324 and generalizations 

4.1 The pattern 1324 

This pattern has been considered in [3j [13]. In [3], Dotsenko and Khoroshkin give a recurrence 
for its cluster numbers r n fc. This recurrence, which involves the Catalan numbers, is essentially 
equivalent to our derivation of equation (fl~2]) below. In [IS], Liese and Remmel use a technique 
developed in [H] to obtain an ordinary generating function that is equivalent to i?i324(— 1, x). Here 
we find the differential equation satisfied by the bivariate generating function o;i324(n, z). 

Theorem 4.1. Let oj{z) = Wi324(u, z). Then uj is the solution of 

zuj^ - {{u - l)z - 3)u/ 4 ) - 3(n - l)(2z + l)o/ 3 ) + (u - l)((4n - 5)z - 6)u/'+ 
{u - l)(8(u - l)z - 3)u/ + 4(n - l) 2 zu = 0, 

with £j(0) = 1,^/(0) = -l,w"(0) = oj( 3 )(0) = 0. 

Proof. In a cluster (71"; h, . . . , ik) with respect to 1324, we have ij + \ — ij £ O1324 = {2, 3} for all j. 
Consider first /c-clusters of length n where ij + \ — ij = 2 for all j, i.e., (ii, . . . , if.) = (1, 3,5, ... ,2k — 
1) € Z? k , where n = 2k + 2. The poset in corresponding to this choice of indices is drawn in 
Figure [2] (when k = 1, this poset is just a chain). The number of linear extensions of this poset is 
the Catalan number Cj- = ^rpy ( 2 ^) , since it equals the number of standard Young tableau of shape 
2 x k (see for example [15]). 
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7T1 



Figure 2: The order relationships in fc-clusters with respect to 1324 where neighboring occurrences 
overlap in two positions. 



A cluster ■ ■ ■ ,ik) may contain two neighboring occurrences of 1324 that overlap in only 

one entry, i.e., ij+i — ij = 3 for some j. In this case, that entry 7r^ +1 is larger than all the entries 
of 7r to its left and smaller than all the entries to its right. In general, a poset in V°u consists 
of a tower of pieces isomorphic to the poset in Figure [2j where the top element of each piece is 
identified with the bottom element of the piece immediately above. A linear extension of the poset 
is uniquely determined by giving a linear extension for each one of the pieces, since there are no 
incomparable elements in different pieces. It follows that the OGF for the cluster numbers is 

^ 1324(t ' x) = i - E fc >x X c k t"x^ - x = i + .-'cw - x (12) 

_ x(l - 2tx(l + x) + a/1 - 4tx 2 ) 
~ 2(1 -tx{l + x) 2 ) X ' 

where C(x) = Z k > C k x k = 

To be able to apply Theorem 11.11 we need the exponential generating function for the cluster 
numbers. From equation (I12p and the fact that the generating function for the Catalan numbers 
satisfies C{x) = 1 + xC(x) 2 , we deduce that A(t,x) = 1 — x — Ri324:(t,x) satisfies the algebraic 
equation 

(1 - tx(l + x) 2 )A(t, x) 2 + (2tx(l + x) + x - 2)A(t, x) - tx - x + 1 = 0. 

It follows that A(t, x) satisfies a linear differential equation, which we have found using the Maple 
package gfun: 

d - 

(4t 2 x 5 + 8t 2 x 4 + (4t 2 - t)x 3 - 6tx 2 -tx + l)x—A(t, x) 

ox 

+ (4t 2 x 5 - 4t 2 x 3 - 2tx 3 + Qtx 2 - l)A(t, x) + {At 2 + 2t)x 3 - 7tx 2 + 1 = 0. 
Applying the operator L to the above equation and using Lemma 11.21 we get 

(4t 2 / 5 + 8t 2 I 4 + Ut 2 - t)I 3 - 6tl 2 -tl + \)z^-A(t, z) 

oz 

3 2 

+ (4t 2 / 5 - 4t 2 / 3 - 2tl 3 + 6tl 2 - l)A(t, z) + (4t 2 + 2t)^- - It*- + 1 = 0. (13) 
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Differentiating (|13p four times with respect to z, we obtain a differential equation satisfied by 
A{z) = A(t,z), namely 

zA^ - (tz - 3)A^ - 3t(2z + 1)A^ + i((4t - \)z - 6)A" + t(8tz - 3) A' + 4t 2 zA = 0, 

with initial conditions A(0) = 1,A'(0) = -1,A"(Q) = 0,^( 3 )(0) = 0. Making the substitution 
t = u — 1 and using Theorem ll.il we obtain an equation for uj. □ 



4.2 The pattern 134 ... (s + l)2(s + 2)(a + 3) . . .m 

The method that we used to find a differential equation satisfied by ^1324(^,2:) can be generalized 
to the pattern a = 134 . . . (s + l)2(s + 2)(s + 3) . . . m for arbitrary s > 2. Note that for m > 2s, 
a is a chain pattern and Corollary 12.61 applies. Thus, in this section we consider only the case 
s + 2 < m < 2s. 

Theorem 4.2. Fix s + 2 < m < 2s, and let a = 134. . . (s + l)2(s + 2)(s + 3) . . . m. Then 

R (tx) = - 1) (M) 



where 



Proof. In any cluster (ir; i\, . . . , ik) with respect to a, we have — ij £ O a = {s, s + 1, . . . , m — 1} 
for all j. The overlaps that create incomparable elements in the poset are between neighboring 
occurrences of a that share m — s entries. Let us first consider fc-clusters of length n= (k — l)s + m 
where ij+i — ij = s for all j, which we call dense clusters. The poset in V° k giving the order 
relationships in dense fc-clusters is drawn in Figure [3l The number of dense £;-clusters is the 
number of linear extensions of this poset, which we call Qt- Note that this number does not change 
if we erase the bottom element and the top m — s — 1 elements. The linear extensions of the 
resulting poset Qk are in bijection with lattice paths from (0, 0) to (ks, 0) with steps u = (1, s — 1) 
and d = (1, —1) that do not go below the x-axis. We call these s-Dyck paths, and note that 2-Dyck 
paths are just standard Dyck paths. Indeed, to construct the path corresponding to a given a linear 
extension, read the elements of Qk in the increasing order given by the linear extension. For each 
element that is circled in Figure El draw an up-step u, and for each of the other elements, draw a 
down-step d. An example of this bijection is given in Figure HI 

An s-Dyck path L can be uniquely decomposed as L = uL\dhid . . . L s _\dL s , where the Lj are 
s-Dyck paths. It follows that the ordinary generating function B(x) for s-Dyck paths, where the 
exponent of x is the number of up-steps, satisfies the equation 

B(x) = l + xB(x) s . (15) 

It is a standard application of Lagrange inversion to deduce that the number of s-Dyck paths with 
k up-steps is 

1 fsk^ 



(s - l)k + 1 V kj' 

We conclude that the OGF for dense clusters with respect to a, or equivalently, linear extensions 
of posets Qk where k can vary, is x m ~ s {B(tx s ) — 1). 
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7I "(fc-l)s+m 




Figure 3: The poset Qk in 7? 1 34...(s+i)2(s+2)(s+3)...m correS p 0nc ling to dense £;-clusters. In this 
picture, s = 5 and m = 10. 



In general, since neighboring occurrences in a cluster can overlap in any number between 1 and 
m — s of positions, every cluster consists of a sequence of dense clusters, where neighboring dense 
clusters overlap in any number between 1 and m — s — 1 of positions. The corresponding poset 
consists of a tower of pieces isomorphic to Qk for some k > 1, where the j largest elements (for 
some 1 < j < m — s — 1) of each piece are identified with the j smallest elements of the piece 
immediately above. Since there are no incomparable elements in different pieces, a linear extension 
of the resulting poset is determined by the linear extensions of the pieces. The OGF for the cluster 
numbers follows. □ 

From the expression in Theorem 14.21 we can proceed as we did for the pattern 1324 to obtain 
the EGF for the cluster numbers: we use equations (|14|) and f)15j) to get an algebraic equation for 
A(t, x) = 1 — x — R a (t, x), which we first turn into a differential equation for A(t, x) and then, using 
Lemma ll.21 into one for u a {u, z) = A(u — 1, z). 

Example 1. Let a = 13425, which is the case s = 3,m = 5. From Theorem 14.21 we obtain using 
Maple that A(t, x) = 1 — x — R a (t, x) satisfies a differential equation of the form 

d 2 - d - 8 

p (t,x)x 2 -^A(t,x) + p 1 (t,x)x—A(t,x) +p 2 (t,x)A(t,x) + s ^c i (t)x l = 0, 

where the pi(t,x) and the Cj(t) are polynomials, and the pi(t,x) have degree 11 in x. In general, 
the order of this differential equation is at most s — 1, by |18t Theorem 6.4.6]. Applying the 
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25 



2 



¥ 1 



Figure 4: An example of the bijection between linear extensions of Q§ and lattice paths, for s = 5 
and k = 5. 

transformation L and using Lemma ll.2l we obtain the following equation for A(t,z): 



Differentiating 11 times with respect to z, we get a linear differential equation for A(t, z), and hence 
also for uj a (u, z). It is a differential equation of order 13 with polynomial coefficients. 

It would be interesting to determine the smallest order of a differential equation satisfied by 
w a (u, z) for arbitrary values of s and m. 

5 Other patterns of length 4 

5.1 The pattern 1423 

As in the case of the pattern 1324, we have that O1423 = {2, 3}. In this case, for each k > 1 there 
is a unique fe-cluster (jr; i\, . . . , ik) of length n = 2k + 2 where — ij = 2 for all j, because the 
poset P^ 4 ^ 3 ife is a chain 7Ti < ir 3 < ir 5 < ■ ■ ■ < ir 2 k+i < ^2k+2 < • • • < vr 4 < 7r 2 . 

A general A;-cluster consists of blocks of marked occurrences that overlap in two positions as 
above, where the last occurrence in each block overlaps the first occurrence of the next block in 
one position (so that — ij = 3). If the first such block has k\ occurrences of 1423, then, in the 
corresponding poset, the block forms a chain with 2k\ + 2 elements. The element just above the 
middle of the chain is 7^/^+2, which is also the first entry of the second block, and thus the bottom 
element of another chain with 2k 2 + 2 elements. The poset of order relationships satisfied by the 
entries of the cluster is drawn in Figure 

It follows that the cluster numbers satisfy the recurrence 




A{t, z) + Pl (t, I)z—A(t, z) + p 2 (t, I)A(t, z) + V Ci(t)- = 0. 

oz l\ 



i=l 




n/2 



(16) 



i=2 



16 



fci+i 




Figure 5: A generic poset in V 



with initial condition r^o = 1 and r^j = for i < 3 in all other cases. This recurrence was found 
by Dotsenko and Khoroshkin |3j. Multiplying (|16p by (— l) k on both sides, summing over all k, 
and letting s n = X^(— l) fc 7"n,fc> we obtain the recurrence 



with si = 1 and = for i < 3 in all other cases. 

If we let S(x) = 1 + Yl n >i s nX n = 1 + x + i?H23(— 1, x), recurrence (fT7|) is equivalent to the 
functional equation 



Although it is straightforward to expand this equation to recover (fT7|) . we had to use a generating 
tree with three labels for the set of clusters to find this equation. Backed by numerical computations 
by Mireille Bousquet-Melou using the Maple package gfun, we conjecture that S(x) is not D- 
finite, that is, it does not satisfy a linear differential equation with polynomial coefficients. This is 
equivalent to the following statement. 

Conjecture 5.1. The generating function 1^1423(0, z) is not D-finite. 

This conjecture is interesting for two reasons. On one hand, proving it would give the first 
known instance of a pattern a for which Wo-(0, z) = 1/P a (0, z) is not D-finite. On the other hand, 
it suggests that a related conjecture for classical patterns (i.e., where occurrences are not restricted 
to consecutive positions) may be false. Noonan and Zeilberger |16| . extending a speculation of 
Gessel [9], conjectured that for any classical pattern, the generating function for the number of 
permutations avoiding it is D-finite. Note also that P a (0,z) is already not D-finite for a = 123 (in 
the consecutive case), since its denominator has a factor cos(3z/2 + vr/6) [7], hence ^123(0,^) has 
infinitely many singularities. 




n/2 



(17) 




(18) 
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Similar posets to the one in Figure [5] arise when considering the patterns 154263 and 165243. 
If a is either of these patterns, then O a = {3,5}, and for clusters where — ij = 3 for all j, 
P^ ix i is a chain of length n = 3k + 3 where 7^+3 has exactly k + 1 elements strictly below it. 
An argument analogous to the one for the pattern 1423 implies that for every n and k, each poset 
in "P^ 263 is isomorphic to exactly one poset in 'P^ 243 , and thus 154263 and 165243 are strongly 
c-Wilf-equivalent. This proves another conjecture of Nakamura |15j . 



We finish this section with a curious connection between occurrences of the pattern 1423 and 
occurrences of a family of patterns. The poset in Figure [5] suggests a close similarity between 
clusters for 1423 and clusters for the infinite set of patterns 

E = {12 ... a(a + 2) (a + 3) . . . (2a) (a + 1) : a > 2} = {1243, 123564, 12346785, . . . }. 

Note that any two of these patterns can only overlap with each other in one position, and that in 
two adjacent occurrences, the 'a + 1' of the left occurrence is the '1' of the right occurrence. 

To be precise, suppose that a A;-cluster with respect to 1423 consists of blocks of k%, &2, . . . , kh 
occurrences (with kj > 1 for all j), where adjacent occurrences within a block overlap in two 
positions and the last occurrence of each block overlaps the first occurrence of the next block in one 
position. This poset, which is drawn in Figure is isomorphic to the poset of order relationships 
of an fa-cluster with respect to E where the jth pattern is 12 . . . kj(kj + 2)(kj + 3) . . . (2kj)(kj + 1). 
It follows that r 4423 = r^ h when k and h are related by 2k + h + 1 = n, and that r^ h = if n — h 

is even. Denoting r 4423 (/j) = J2k r rfk 3 ^ k ano ^ r nW = 2~2h r nhf h ' we nave 



r S/,N _ S .h _ r 1423 t h - r 1423,n-2fc-l _ .n-1 1423/ 1 

r n\ l ) — n > h ~ Z^i r n,(n-h-l)/2 l ~ Z^i n > k ~ n ^ ■ 

h=l h=l 

n — h odd n — h odd 

if t 7^ 0, so 

MM) = = ^ 423 (i)^ = i/wi,t*). 

In particular, the generating functions enumerating occurrences of these patterns are related by 

Ps(u,z) = — — j . 

u-2 + 



PU23 + !,(«- 

It follows, for example, that Pz(2, z) = ^1423(2, z), and that ws(0, z) + ws(2, — z) = 2. 



5.2 The pattern 2143 

Again we have that O2143 = {2, 3}, and so A;-clusters with respect to 2143 can be broken into blocks 
in such a way that inside each block, adjacent occurrences overlap in two positions, and each block 
overlaps with the next in one position. As in the case of the pattern 1423, the poset corresponding 
to a block with kj occurrences is a chain of length 2k j + 2. For example, if k\ = i — 1, we get the 
chain 1T2 < 7i"i < ir^ < -K3 < • • • < 7^ < K2i-l- The second largest element of each chain is the 
second smallest element of the next chain, corresponding to the position where a block overlaps 
with the next one (see Figure [6|) . 
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Figure 6: A generic poset in "P 2143 . 



The smallest element of each chain has the property that it does not cover any other elements in 
the poset. The elements other than the bottom one with this property will be called feet. Compare 
this poset with the one in Figure [5l which had no feet. Let r n /- 1 be the number of /c-clusters of 
length n where tt\ = £ + 2. Such a cluster corresponds to a linear extension of a poset where exactly 
£ feet precede tx\. 

For these refined cluster numbers, one obtains the recurrence 

r n ,k,£ = 5 n , k ,i + ^2 ^2 (n-2i- j)(£ + l)( _ J r n - 2 i+l,k-i+l,j , (19) 

i=2 j=e-i \ 1 J 

where 8 n k,£ equals 1 if n = 2k + 2 and £ = 0, and otherwise. An equivalent recurrence was 
given by Dotsenko and Khoroshkin [3]. To obtain (|19p . note that the poset P in Figure [6] is either 
a chain with n = 2k + 2 elements (and thus with no feet), or it consists of a chain C with 2i 
elements attached to the poset P' corresponding to the (k— i+l)-cluster ir2iTT2i+i ■ ■ Given a 
linear extension of P' having j feet preceding 7T2i, to produce a linear extension of P having £ feet 
preceding %\ we have n — 2i — j choices for the value of iT2i-i, £ + 1 choices for the value of 112 (since 
it has to be one of the £ + 1 values lower than 7i"i), and { 2t+ 2i-4~^) wavs to interleave the 2i — 4 
elements of C and the j — t + 1 elements of P' whose value is strictly between 7Ti and 7T2i- 

6 Asymptotic and analytic results 

6.1 Analytic properties of u a 

Recall that 

u a (u,z) 



P a (u,z) 

is the inverse of the generating function counting occurrences of a. In all the cases we have been 
able to solve, u) a (0,z) is an entire function in the complex plane, so that the counting generating 
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function P a (Q,z) is a meromorphic function and its dominant singularity is the smallest positive 
zero of w CT (0, z). We do not know if this is always the case (below we show an example where w CT (2, z) 
is not entire), but we can prove it in several interesting cases. Since u a (u,z) = 1 — z — R a (u — l,z), 
this is naturally related to the growth of the cluster numbers r n ^. Recall that V° is set of all posets 
associated to clusters of size n with respect to a. 

Theorem 6.1. Let a E S m . Suppose that there exists a > so that, for all n, all posets in 
contain a chain of length at least an. Then, for every fixed u £ C, uj a {u,z) is an entire function 
of z. 

Proof. A poset in is determined by the positions ,ik) of the occurrences of a in a k- 

cluster. Since i\ < ■ ■ ■ < i^, the total number of such tuples (where k is not fixed) is at most 2 n . 
By hypothesis, each poset has a chain of length an, and the number of linear extensions is at most 
the number of ways of deciding the relative order of the n — an elements not in the chain. Each 
such element has at most n possible placements relative to the elements of the chain and the other 
elements that have already been placed. This gives an upper bound n n ~ an for the number of linear 
extensions of each particular poset. Summing over all clusters of size n with respect to a we obtain 



n„n-an 



Using the bound n\ > n n /e n , we get 



n! / ra L 



which tends to as n goes to infinity. 

To prove that R a (t, z) = ^ r n ^t k ^ is entire for any t € C, it is enough to show that 

lim | ^k r n,kt k \ ' = q_ 



< 



n— >oo \ n\ 

Using that 

\t\ n Ek r n,k if |*| > 1; 

v Efc r «,fc if 1*1 < !> 

the result follows from the previous bound. □ 

The above theorem applies trivially to chain patterns, and more generally to patterns with 
u\ = 1, as well as to non-overlapping patterns. 

Corollary 6.2. If a G S m is a chain pattern, then for any fixed u, u a (u,z) is an entire function 
of z. 

Proof. Every poset in is a chain of length n, so Theorem 16.11 applies with a = 1. □ 

Corollary 6.3. If a £ S m satisfies o\ = 1, then for any fixed u G C, u) a (u, z) is an entire function 
of z. 
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Proof. Let (ir;ii, . . . ,i k ) be a /c-cluster with respect to a. Since ir^ is the smallest element in the 
occurrence of the a starting in position ij, we have 

Til < 7r i2 < " " " < ^fc • 

Since cr has length m, this gives a chain in the associated poset of length k > (n— 1) /{m — 1) > ro/m, 
so Theorem 16,11 applies taking a = 1/m. □ 

Corollary 6.4. For any non- overlapping pattern a £ S m and for every fixed u G C, oj a (u,z) is an 
entire function of z. 

Proof. Let a = a\ and b = o~ m , and note that a ^ b. Without loss of generality we can assume 
that a < b, since a and its reversal have the same cluster numbers. We know from Section [3] that 
fc-clusters with respect to a have length n = k{m — 1) + 1, and Figure DJii) shows that every poset 
in Vn contains a chain of length m + (b — a)(k — 1). Since b — a < m — 1, we have that 

7i — 1 b — n 

m + (6 - a)(k - 1) > 1 + (b - a)k = 1 + (b - a) > n. 

m — 1 m — 1 

Thus, Theorem 16.11 applies with a = (b — a)/(m — 1). □ 

We conclude this section by showing that ui a (u,z) may fail to be entire. Out of the seven 
equivalence classes for consecutive patterns a of length four, five of them have u\ = 1, namely 
1234, 1324, 1423, 1342 and 1243. By Corollary 16.31 oj a (u,z) is always an entire function of z for 
these patterns. For the pattern a = 2143, the argument in Section 15.21 shows that every poset in 
Vn 143 contains a chain of length at least ra/3, hence ui2U3{u,z) is entire as well by Theorem 16.11 
Perhaps surprisingly, this is not true for the remaining length four pattern 2413. 

Proposition 6.5. The function 6^2413(2, z) is not entire. 

Proof. Note that O2413 = {2,3}, and consider clusters ,i2£) with respect to 2413 where 

ij + i — ij equals 2 for odd values of j, and it equals 3 for even values of j, that is, (h,i2, ■ ■ ■ ) = 
(1, 3, 6, 8, 11, 13, 16, . . . , U - 4, U - 2). Such a cluster has size n = 5£ + l, and the poset P^_ jht 
is drawn in Figure [71 We claim that this poset has at least £\ 5 linear extensions, since the poset 
can be partitioned into 5 horizonal layers (4 of size I and one of size i + 1) so that the elements 
within each layer are incomparable. It follows that for n = 5£ + 1, 

,2413 ^' - n 



using Stirling's approximation. Hence, disregarding polynomial terms in n, 

( E k r n , k \ 1/n y 1 
V n\ ) ~ 5' 

and the radius of convergence of R2&iz(l,z) is finite. Thus ^2413(2,2;) = 1 — z — ^2413(1, z) is not 
an entire function. □ 

Note that this argument does not disprove that i?24l3( — 1> z) (equivalently, W24i3(0,z)) is entire, 
and in fact it is possible that, with all the cancelations that take place in i?24i3( — 1> z), its coefficients 
are much smaller. 
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Figure 7: A poset in V 
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6.2 Asymptotic behavior of a n (a) 

The first result in the literature regarding the asymptotic behavior of the number of permutations 
avoiding an arbitrary consecutive pattern is the following. 

Proposition 6.6 ([B]). For every a € S m with m > 3, there exist constants < c < d < 1 such 
that c n n\ < a n (a) < d n n\ for all n. Additionally, WiHn^,^ (a n (o")/n!) 1/ ' n exists, and it is strictly 
between 0.7839769 and 1. 

The value of this limit, which is called the growth constant of the pattern and is sometimes de- 
noted p a , was determined in [7] for several patterns. The following conjecture, which is equivalent 
to the fact that the monotone pattern has the largest growth constant among patterns of the same 
length, is still open. 

Conjecture 6.7 (|7J). For every a £ S m there exists uq such that a n (a) < a n (12...m) for all 
n > uq. 

Our main result in this section is a proof of this conjecture for non-overlapping patterns. We 
remark that Bona [2] has shown that the proportion of non-overlapping patterns is greater than 
0.364. Hence we verify the conjecture for a positive fraction of all patterns of each length. 

Theorem 6.8. Let m > 3 and let a G S m be a non- overlapping pattern. Then there exists uq such 
that 

a n (a) < a n (12 ...m) 

for all n > uq. 

Proof. We prove that the growth constants for these patterns satisfy p a < PYi...m' This is clearly 
equivalent to the statement of the theorem. 

Let ^rnon( z ) — w i2...m(0, z) and Lo no i(z) — uj a {§,z). These are both entire functions by Corol- 
laries 16.21 and 16.41 It follows that the singularities of the generating functions 



V]a n (12...m)— and — = V]a n (cr) — 

„ n! Unoi{z) V n] 
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are poles at the zeroes of ui mon {z) and ui no i(z), and that their radii of convergence are l/p\%.. m 
and I/pa, respectively. Since the coefficients of these generating functions are non-negative, Pring- 
sheim's Theorem |8 ; , Theorem IV. 6] implies that they have real singularities at z mon = l/px2... m 
and z no i = I /p a - These values are the smallest positive zeroes of uj mon (z) and ui no i(z), respectively. 
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Our goal is to show that z mon < z no \. Since u) mon (z) and uj no i(z) are continuous functions on M 
with Lu mO n(0) = w no ;(0) = 1, and we know by Proposition 16.61 that 1 < z mon ,z no i < 1.276, it is 
enough to show that u mon (z) < uj no i(z) for 1 < z < 1.276. 

In the rest of the proof, we assume that 1 < z < 1.276. For convenience, we also assume that 
m > 4, since the case m = 3 is proved in [TJ. By equation flSJ), 



^) = £?^-E f,w 1 v <l - z+ -- — + 



o (jm)! i ^(jm + l)! m! (m + 1)! (2m)!' 

since each negative term of the alternating sum is larger in absolute value than the following positive 
term. On the other hand, using that A;-clusters with respect to a have length n = k(m — 1) + 1, 
and letting d k = r fc ( m _i) + i ifc , we have 

_^ „fe(m— 1)+1 gin ^ gk(m— 1)4-1 

w noi (z) = 1 - ar - y)(-l) fc dfc7T7 ,s , n , > 1 - z + — r - V 4777 7— -77, 

g (fc(m-l) + l)! m! (fc(m-l) + l)! 

fc even 



since the 4 are positive and di = 1. Thus, it suffices to show that 

^ gk(m - 1)+1 gin+l z^ m 

^ dk (h(m -1U11I < I'm 4-1 V ~ TWlT ^ 21 ) 



'(Jfe(m-1) + 1)! (m + 1)! (2m)! 

A; even 

for 1 < z < 1.276 and m > 4. 

Let a = o"i and 6 = a m , and assume without loss of generality that a < b. When k = 2, the 
poset in Figure [Tfi) has 4 = ("n_"i 2 ) ( 2 'm-V' > ) linear extensions. To find an upper bound on d 2 , 
note that for any fixed value of a + b, the two binomial coefficients in this product are maximized 
when 5 = o + l. On the other hand, it is an exercise to show that max,, ( 2a ~}) ( 2m - 2a ~ 1 \ = ( 2m_3 ) 

' "V a— 1 ) V m— a— 1 / V m— 2 I ' 

and it is attained when a = lora = m- 1. It follows that d 2 < ( 2 ™J 2 3 )- A similar 

reasoning shows 

that for every k, 4 is maximized when a = 1 and b = 2 (or, by symmetry, when a = m — 1 and 
b = m). We obtain the bound 



4 < 



fc(m - 1) - l\ / (k - l)(m - 1) - 1\ /2(m - 1) - 1\ _ (m- l)(Jfe(m - 1) - 1)! 
m-2 J\ m-2 / "\ m-2 J~ (m - l)! fc (/c - 1)! 



the right hand side being the number of linear extensions of the poset in Figure [TJi) when a = 1 
and 6 = 2. This bound implies that 

fc(m— 1) j / z m ~ * \ ^ 1 / Z m— ^ 

477-7— ^TT7< 77 7— TT7 < 



'(fc(m-l))! - fe! V("»- 1)!/ - 2\(m-l)l 
for k >2, and so 



„fc(m— 1)+1 ^ ^ ^k(m— 1) 



S dfe (Jfc(m - 1) + 1)! ~ 2m - 1 ^ dh 



< 



(k(m- 1) + 1)! - 2m- 1 f-^ (fc(m-l))! 2m-l z ^VO- 1 ) ! 



z m-l 



fc even fc even 



„2m-l 



(2m-l)(m- l)! 2 (1 " 



(m-l)! 2 _ 
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We have reduced the proof of inequality (f2T|) to showing that 



1 



z 



,m— 1 




(2m)!' 



Since z 



2(m i) ^ ^ m _ iy?^ ^ e above inequality is equivalent to 



,m— 1 ~2(m— 1) ~3(m— 1) ~m— 2 



(m + 1)! (2m)! (m - l)! 2 (m + 1)! (m - l)! 2 (2m)! (2m - l)(m - l)! 2 



> 



for 1 < z < 1.276 and m > 4. This is easy to verify, since the first term, which is positive, dominates 



7 Open problems 

In analogy with Conjecture 16.71 Nakamura |15l Conjecture 2] conjectures from numerical evidence 
that the pattern 123 . . . (m — 2)m{m — 1) (which is non-overlapping) is the hardest to avoid among 
all patterns of length m. A special case of this conjecture is that this pattern is harder to avoid 
than any other non-overlapping pattern. It also appears to be the case that the pattern 134 . . . m2 
is the easiest to avoid among non-overlapping patterns of the same length. The last two conjectures 
can be combined as follows. We expect that the ideas in the proof of Theorem 16.81 including a 
detailed analysis of the coefficients dk, may be useful in proving this conjecture. 

Conjecture 7.1. For every non-overlapping a G S m , there exists uq such that, for all n > uq, 



We have seen in Section [6.11 that u a (u, z) is an entire function of z for a large class of patterns. 
On the other hand, ^2413 (2, z) is not entire. However, it is still plausible that u a (0, z), which is the 
inverse of the generating function for permutations that avoid a, is always an entire function. 

Question 7.2. Is cj ct (0, z) an entire function for every pattern a? 

To conclude, let us mention a recent result of Ehrenborg, Kitaev and Perry [5], proved using 
methods from spectral theory, and previously conjectured by Warlimont [19] . 

Theorem 7.3 ([5]). For every pattern a, a n {a)/n\ = 7/5" + 0{r n ), where 7, p and r are positive 
constants such that p > r. 

It would be interesting to find a more combinatorial proof of this important result using sin- 
gularity analysis of generating functions. If Question 17.21 is answered positively, and one can show 
that the zero of w CT (0, z] of smallest modulus is always unique and simple (we know that this is the 
case for several patterns) , then the tools from [HI Chapter IV] would give a proof of Theorem 17.31 
without using spectral theory. 



all the other terms. 



□ 



a n (123 . . . (m - 2)m(m - 1)) < a n (a) < a n (134 . . . m2). 
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